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ABSTRACT 



This document examines whether early targeted academic 



intervention can improve college admission test scores and pre- college 
planning for students identified through the Duke Talent Search and the 
school system Challenge Program. Students selected for the Early Awareness 
Test Preparation Pilot Program were invited to participate during their 
eighth grade year and a 4 -year commitment to the program was requested. 
Through the pilot program, students annually attended a series of classes in 
Math, English, and test taking skills. Preliminary results indicated that the 
program assisted a majority of students in gaining an opportunity to increase 
test scores. Both the lack of an exit summary and the size of the pilot 
v program were drawbacks to the study. Approximately half of the students were 
unable to attend the last testing. It is hoped that upon completion of the 
4-year pilot, results from the first year can be assessed in comparison to 
the total remaining student population. This could provide a clearer picture 
of what can be accomplished through the utilization of the program. (Contains 
13 references.) (JDM) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Test Prep 1 



r- 

r- 

n 




Running Heading: Early awareness test preparation pilot program 



Early Awareness Test Preparation Pilot Program 
Mary-Beth Muskin, Creighton University 
Sharon Cipperley Friedlander, Omaha Public Schools 



BEST COPY AVAILABLE 



CO 

LO 

C\J 



U.S. DEPARTMENT OF EDUCATION 
Office of Educational Research and Improvement 

EDUCATIONAL RESOURCES INFORMATION 
CENTER (ERIC) 

□ This document has been reproduced as 
received from the person or organization 
originating it. 

□ Minor changes have been made to 
improve reproduction quality. 



PERMISSION TO REPRODUCE AND 
DISSEMINATE THIS MATERIAL HAS 
BEEN GRANTED BY 




• Points of view or opinions stated in this 
document do not necessarily represent 
official OERI position or policy. 

V ...... ... J 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERIC) 



2 



1 



Test Prep 2 



Abstract 

This article examines whether early targeted academic intervention can improve college 
admission test scores and pre-college planning for students identified through Duke 
Talent Search and the school system Challenge Program. Students selected to participate 
in the Early Awareness Test Preparation Pilot Program were invited to participate during 
their eighth grade year and a four year commitment to the program was requested from 
each student. Through the pilot program students attended a series of classes annually 
that addressed instruction in the areas of math, English and test taking skills. 
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Early Awareness Test Preparation Pilot Program 

The Early Awareness Test Preparation Pilot Program was designed as a self- 
supporting program to establish if early academic intervention (starting in grade 8) 
would enhance ACT/SAT I test scores. The Early Awareness Test Preparation Pilot 
Program was designed for the purpose of improving college admission test scores and 
pre-college planning. The acceptance of SAT I results as the leading indicator of one’s 
overall ability and worth is contrary to College Board and testing services guidelines, but, 
unfortunately, states and individual schools find themselves ranked and rated 
according to their student SAT scores (Wiggins, 1992). 

The Early Awareness Test Preparation proposal was developed in support of the 
district’s Annual Plan. Additionally, it addressed voiced concerns of some parents about 
the inability of their children (students ranking in the top 24%) to get into their first 
university of choice. Upon review of the situation with university personnel on an 
informal basis, ACT/ scores were cited as the main single denominator for this segment 
of bright student’s denial of admission. The Early Awareness Test Preparation Pilot 
Program proposed this study to evaluate whether there is an impact on test scores 

for this population of students, if they have the opportunity to work on test taking skills. 
Research is mixed on whether coaching for the test improves student scores. “The has 
ETS abandoned its original opposition to coaching (Omstein 1993). However, there are 
indications that the additional work would, increase top student’s chances of admission 
to their colleges of choice. 
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An extensive review of the literature revealed that there has been only one such 

attempt in the past. In 1984 the State of Louisiana ranked next to the bottom of all states 
in a Department of Education report on national ACT and SAT scores (Chaika 1985) 

The state’s response was to devise a course to teach SAT test-taking skills to junior high 
students to see if gifted seventh graders taking the Duke Talent Search would score 
higher on the SAT than did the previous class that had not taken the course. 

The study was small, and focused on seventh grade students. Preliminary results 
indicated that students who regularly attended classes increased their scores thus 
qualifying for the Duke Talent Search Program (Chaika, 1985). 

Additional research by Carris (1995) states that if students are offered a test- 

prep program, it should be a quality program offered early in their academic careers. Her 
recommendations indicate a preference for starting at kindergarten and infusing test 
preparation into the curriculum on a daily basis. 

The overall goal of the Early Awareness Test Preparation Pilot Program was to assess 

whether early, ongoing intervention can have an effect on increasing test scores. 

Peltier (December 1989) points out that theoretically there are three ways to improve test 
scores: 

• genuine improvements in the abilities measured by the test 

• decreased anxiety, increased familiarity with the format 

• heightened test taking skills 
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Peltiers (December 1 989) three points are programmatically addressed in the Early 
Awareness Test Preparation Pilot Program. Additionally, the weight of the evidence 
clearly indicates that improving one’s score by study and effort is not only possible but 
also probable. (Peltier, 1989). 

Method 

Participants 

The target population included students who had been selected through the Duke 
Talent Search Program to take the ACT or SAT I as seventh graders and who had 
completed the tests. Student selection was based on the ACT/SAT 1 scores of students 
that participated in the Duke Talent Search program achieving state recognition. 
Additionally, students who successfully completed the SOI (district administered 
intelligence test) and had been recommended through the district challenge program also 
were invited to participate. Initially, a total of 90 students were invited to participate in 
the program. Of the total number of invited participants, 38 enrolled in the program 
during the first year. The enrolled student population was comprised of 30 students from 
the Duke Talent Search Program and an additional 8 who were identified through the SOI 
and challenge teacher recommendation. Students participating in the Early Awareness 
Test Preparation Pilot Program were identified from seven middle schools which 
represented a cross-section of students from this mid-size city. Minorities comprised 
1 3% of the total participating student population. 
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Table 1 



Program Components 

The Early Awareness Test Preparation Pilot Program was presented to parents and 
students as a four-year commitment both on the part of the school district and the 
students. The program effectiveness was evaluated annually with a comprehensive 
evaluation at the end of the four-year commitment. Program start-up funds were 
procured through the acquisition of a local grant. 

The curriculum for the program was created using a developmental model. The focus 
was on skill building in the content areas with a section on general test taking and critical 
thinking skills included. Prior to the start of the program, a parent meeting was held to 
assist parents in understanding annual goals and to actively involve them in the process. 
During the eighth grade year, classes focused on the sections found on the ACT/ SAT I. 
These sections included grammar, rhetoric, reading and math concepts. Specialists in 
each field were contracted to teach the section falling within their expertise. Instructors 
who were selected for the program were told that they were to develop the curriculum 
for each year of instruction and that lessons should be based on program goals which 
were tied to the desire to assess whether this early intervention could boost student scores 
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on the ACT/SAT 1. A study indicated that there were positive gains when there was an 
application of game playing strategies with high school seniors who were taking a test- 
taking course (Wiggins, 1992). A goal of the pilot program was for 
instructors to make lessons meaningful yet fun. Classes ran from October 1997 through 
May of 1998. During the designated week of class, three classes were offered for a total 
of five sessions for each set of class sessions. All students participating in the program 
received a total often hours of instruction. Pre- and post-tests taken were given to 
assess student progress over the year and to evaluate areas of strength and weaknesses. 
Additionally, a survey was administered to parents and students which requested their 
input on program strengths and weaknesses. Results from the surveys were used to 
determine what program changes were needed for the following year. 

A meeting was held for parents of students participating during there ninth grade year. 
Ninth grade classes focused on review of skills learned relative to test items from the 
post-test and incorporated test taking skills. Classes were conducted from October 1998 
through April 1999. However, based on input from parents the location was changed to a 
centrally located university. This was beneficial because of the centrality of the location 
as well as the student’s exposure to a University setting. 

The plan for years three and four followed a similar format. However, the plan 
required that students be assessed annually to determine course need. Additionally, in 
year three, the sophomore students take the PLAN test and in the fourth year of the 
course students take the PSAT in the fall and the ACT/SAT 1 in the spring. 
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The PLAN and PSAT will assist with diagnosis of strengths and weaknesses of students 
and provide predictor score. 



Results 

Upon completion of the second year of the Early Awareness Test Preparation Pilot 
Program, results for the ninth grade students who remained in the program were positive. 
Thirty-eight students participated in the program during year one and twenty-eight 
participated during year two. Regular class attendance was strong throughout both years. 
However, on the last scheduled class for year one a total of fifteen completed the post-test 
due to a district overlap.in scheduling of events. This unfortunate situation allowed for 
only the total of fifteen student results to be complete in terms of pre- and post-testing 
over the two-year period for comparison purposes. 

The mean score in the verbal area in the fall of 1 997 was 61.1. The mean score in the 
verbal area in the spring of 1998 was 66 and the mean score in the fall of 1999 was 74. 
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Twelve students increased their scores by an average of 19 percentage points. One 
student dropped by one percentage point and the other two went down an average of 1 1 
percentage points. 

The mean score in the area of math in the fall of 1997 was 52, 59 in the spring of 
1998, and 85 in the spring of 1999. Individually all student scores increased with the 
group average being 1 5%. 

In the area of writing skills, students had a mean of 5 1 in the fall of 1 997, but in the 
spring of 1998, the mean went down to 46. However, in the spring of 1999, the student 
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mean increased to 62. Thirteen of the fifteen students achieved an average gain of 15% 
in their scores, while two students had an average decrease of 7.5%. 

Parent/Student Survey ( 1997 - 1998 ) 

Parents and students were surveyed using a question format that provided space for 
comments. Twenty-nine parents and students completed the survey during the first year 
of the pilot program. Results from the surveyed were reviewed with program 
administrators looking for trends in responses. Where concerns were repeated program 
changes were made. Questions from the parent survey along with total responses can be 
reviewed in Appendix A. 

Appendix A 

The comment section revealed student thoughts in a very candid manner. 
Comments included “teachers made the session,” “sometimes I thought hard about what 
we were talking about,” “ classes were fun and I met kids from all over the city.” 
Comments in conjunction with yes no responses indicated that some change was 
necessary in the verbal area. Student statements included “the math was challenging but 
the verbal went too fast,” and “I don’t understand language to well and the instructor 
expected me to know what I was doing.” Scheduling and location changes were made for 
the second year. These changes were based on parent and student statements like, 

“it’s close to me but could we have it somewhere else next year,” “the time was too 

I 

late,” and “sometimes class conflicted with other activities.” 
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Parent/Student Survey ( 1998 - 1999 ) 

The year two surveys were developed with the intent of gaining more specific 
program data. This survey was developed using a Likkert scale that went from A) 
strongly disagree to E) strongly agree. The first page was divided by class topics 
including general strategies, math and verbal. The statements in each section included 

• I attended the (general strategies, math and verbal) portion of the test preparation 
program. 

• The instructor knew the subject well. 

• The instruction will help me prepare for the PSAT/SAT/ACT. 

• The instruction time was too short. 

• The instruction time was too long. 

The second half of the instrument was fill-in the blank. Questions in this section included 

• Name one thing that you learned that will help you with the verbal portion of the test. 

• Name one thing that you learned that will help you with the math section of the test. 

• Name one thing that you learned that will help you on the general test taking/critical 

thinking skills required for the test. 

The data indicated that overall students and parents were happy with the program and the 
location and that changes made in teaching staff were viewed as positive. Informal 
discussion with students that chose not to return indicated that two students had moved 
and that the rest found it hard to continue because of increased school loads and other 
desired activities. 
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Discussion 

Preliminary results indicate that the Early Awareness Test Preparation Pilot Program 
assisted a majority of the students completing pre and post testing in gaining an 
opportunity to increase test scores. The other major contributing factor to success 
included the completion of another school year, which provided some natural growth. 

Major drawbacks to this study included the lack of an exit summary and the size of the 
pilot population. There are a significant number of other school activities that create 
conflicts for students participating in this program. Since this is a district-wide program 
and the scheduling of individual schools varies significantly, there is additional difficulty 
created in finding a time that is good for all students. Approximately half the students 
enrolled and participating in the program were unable to attend the last class (testing) 
during the second year of the pilot, results at this mid-point were based on the 
performance of only fifteen of the remaining twenty-nine students. It is hoped, however, 
that upon completion of the four-year pilot, results from the first year can be assessed in 
comparison to the total remaining student population. Assessment of the total student 
population enrolled in the pilot program would provide a clearer picture of what can be 
accomplished through the utilization of an Early Awareness Test Prep Pilot Program. 
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Racial Composition 



Race 


Raw Number 


Percent of Population 


Caucasian 


25 


64 


African-American 


8 


20.5 


Native American 


2 


5 


Asian 


3 


18 


TOTAL 


38 


100 
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Appendix A 

Parent Survey and Results 

1 . Do you feel intellectually challenged by classes? 

Twenty-nine students responded that they were intellectually challenged. 

2. Did you like going to class? 

Twenty-five students responded that they did like going to class, while three students said 
they did not. 

3. Did you learn new information at the Test Prep class? 

Twenty-nine students responded positively. 

4. Did you feel that the instructors teaching styles met your needs? 

Relative to verbal instruction, nineteen students indicated satisfaction with the instructor 
and ten students were unhappy with the instructor. Relative to math instruction, twenty- 
eight students were satisfied with the instructor while one was not. 

5. Was the schedule flexible enough for your family? 

Twenty-seven students thought that the schedule was flexible enough while two were 
unhappy with the schedule. 

6. Is the present location the best site for a citywide program? 

Twenty-five students liked the location while four did not. 
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